AN INVARIANCE PRINCIPLE FOR THE LAW OF THE ITERATED 
LOGARITHM FOR SOME MARKOV CHAINS 

W. BOLT, A. A. MAJEWSKI, AND T. SZAREK 

Abstract. Strassen's invariance principle for additive functionals of Markov chains with 
spectral gap in the Wasscrstein metric is proved. 



1. Introduction 

Suppose that (E, p) is a Polish space. By B(E) we denote the family of all Borel 
sets in E. By Mi we denote the space of all probability Borel measures on E. Let 
7r : E x 13(E) — > [0, 1] be a transition probability on E. The Markov operator P is defined 
by Pf(x) = J E f(y)n(x,dy) for every bounded Borel measurable function / on E. The 
same formula defines Pf for any Borel measurable functions / > which need not be 
finite. Denote by Bb(E) the set of all bounded Borel measurable functions equipped with 
the supremum norm and let Cb(E) be its subset consisting of all bounded continuous 
functions. 

Suppose that (X n ) n >Q is an E- valued Markov chain, given over some probability space 
(Jl, J 7 , P), whose transition is n and its initial distribution is equal to fiQ. Denote by 
E the expectation corresponding to P. We shall denote by pP, the associated transfer 
operator describing the evolution of the law of X n . To be precise, \xP is defined by the 
formula f E f(x)pP(dx) = f E Pf(x)p(dx) for any / G B b (E) and p G M.\. To simplify the 
notation we shall write (/, p) instead of J E f(y)p(dy). 

Given a Lipschitz function ip : E — > R we define 

S n (iP) := iJj(X ) + + • • • + i)(X n ) for n > 0. 

Our aim is to find conditions under which S n (ip) satisfies the law of the iterated logarithm 
(LIL). This natural question is raised when central limit theorems (CLT) are verified. 
Since 1986 when Kipnis and Varadhan [TT] proved the central limit theorem for additive 
functionals of stationary reversible ergodic Markov chains, it has been a huge amount of 
reviving attempts to do this in various settings and under different conditions (see [T5|[T6"]). 
A common factor of the mentioned results was that they were established with respect to 
the stationary probability law of the chain. In [5] Derriennic and Lin answered the question 
about the validity of the CLT with respect to the law of the Markov chain starting at some 
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point x. Namely, they proved that the CLT holds for almost every x with respect to the 
invariant initial distribution (see also [5] and the references therein). On the other hand, 
Guivarc'h and Hardy [7j proved the CLT for a class of Markov chains associated with 
the transfer operator having spectral gap. Recently Komorowski and Walczuk studied 
Markov processes with the transfer operator having spectral gap in the Wasserstein metric 
and proved the CLT in the non- stationary case (see [12]). Other interesting results under 
similar assumptions were obtained by S. Kuksin and A. Shirikyan (see [L3| 120]). 

The LIL we study in this note was also considered in many papers. There are several 
results governing, for instance, the Harris recurrent chains [21 El [T7] . Similarly the CLT 
results they are formulated mostly for stationary ergodic chains (see for instance [H m 
[221 dHl 122])- In the case when one is able to find the solution to the Poisson equation 
h = f + Ph, the problem may be reduced to the martingale case [5] (see also [17]). But 
the LIL for martingales was carefully examined in many papers (see [TUT [HI [221 [23]) and a 
lot of satisfactory results were obtained. 

Our note is aimed at proving the LIL for Markov chains that satisfy the spectral gap 
property in the Wasserstein metric. It is worth mentioning here that many Markov chains 
satisfy this property, e.g. Markov chains associated with iterated function systems or 
stochastic differential equations disturbed with Poisson noise (see [H]). 

Our result is based upon the LIL for martingales due to Heyde and Scott (see Theorem 
1 in 0). 

2. Assumptions and auxiliary results 

For every measure v G M\ the law of the Markov chain (X n ) n > with transition prob- 
ability 7r and initial distribution z/, is the probability measure F u on (-E N , B(E)® N ) such 
that: 

K[X n +i e A\X n = x]= tt(x, A) and F V [X G A) = v{A), 
where x G E, A G 13(E). The expectation with respect to F u is denoted by E„. For v = 8 X , 
the Dirac measure at x G E, we write just F x and E x . 
We will make the following assumption: 

(HO) the Markov operator satisfies the Feller property, i.e. P(Cb(E)) C Cb(E). 

We shall denote by M.\a the space of all probability measures possessing finite first 
moment, i.e. v G M.\ % \ iff v G M.\ and f E p(x , x)u(dx) < oo for some (thus all) x G E. 
For abbreviation we shall write p XQ ( x ) = p( x o, x )- We assume that: 

(HI) for any v G M.\.\ we have Pv G M.\,\. 

It may be proved that the space A^i,i is a complete metric space when equipped with the 
Wasserstein metric 

d(v x ,v 2 ) = bu P {K/,^) - (/> 2 )| : / : E -> K, Lip/ < 1} 

for ui, V2 G M.i,i and the convergence in the Wasserstein metric is equivalent to the weak 
convergence, see e.g. [21]. (Here Lip / denotes the Lipschitz constant of /.) The main 
assumption made in our note says that the Markov operator P is contractive with respect 
to the Wasserstein metric, i.e. 
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(H2) there exist 7 G (0, 1) and c > such that 

(2.1) d(uP n , vP n ) < C7 n rf(/i, u), for n > 1, fi, v G 

Let /i G From now on we shall assume that the initial distribution of (X n ) n > is 

u. Moreover, 

(H3) there exists Xq G E and 5 > such that 

(2.2) supE^+ 5 (X„)<oo. 

n>0 

It is easy to prove that under the assumptions (H0)-(H3) there exists a unique invariant 
(ergodic) measure /i* G Mi- In particular, /1* G -M^i. The proof was given for Markov 
processes with continuous time in [12] but it still remains valid in discrete case. 

Let no > 2 be such that 

70 = cV° < 1. 

We start this part of the paper with a rather technical lemma. 

Lemma 1. Let g n ^ ■ E 2 ^ k+n ^ — > R for arbitrary k,n > 1, be Lipschitz continuous in each 
variable with the same Lipschitz constant L. Then there exists constant L dependent only 
on L and such that the function 



H n , k (x) = / 7Ti(x,dyx) / ir 2 (yi,dy 2 ) ■ ■ ■ / n 2<yk+n) ^i{y 2{k+n) _ 2} dy 2 {k+n)-\) 

(2.3) y Je Je 

X / 7T 2 (k+n)(y2(k+n)-lAy2(k+n))gn,k(yi, ■ ■ ■ ,V2(k+n)), 



E 

k. 



where 7Ti(yi^i,dyi) = 5 yi l P l (dyi), h > 1 and additionally h > — 1 for all even I, is 
Lipschitzean with the Lipschitz constant L. 

Proof. Define the functions gj : E 3 ' — > R by the formula 

9j{Vo, yi, ■ ■ ■ , i/j-i) = / dj/j) / irj+i{yj, dy j+1 ) x • - • 

X / 7T 2 (k+n)(y2(k+n)-l,<ly2(k+n))gn,k(yx,---,y2(k+n)) for j = 1, . . . , 2(fc + 7l). 



Let Cj t i for j = 1, . . . , 2(k + n) and / = 0, ... ,j — 1 denote the Lipschitz constant of gj 
with respect to y\. Then the Lipschitz constant of H n ^ is equal to £i,o- It is obvious that 
< L for < / < j — 1, j > 1. To evaluate £j,j-i fix yo,yi, ■ ■ ■ ,Vj-2 and yj-i,yj-\. 
Then we have 

0j(yo, i/i, ... , yj-2, j/i-i) - 9j{yo, yi, ■■■ , yj-2, yj-i) 



Kj(yj-iAyj)9j+i(yo,yi, ■ ■ ■ ,llj-i,yj) - / ■K j {y j - 1 ,dy j )g j+1 (y Q iy 1 , . . . 

nAvj-udy^igj+i (yo,yi, ■ ■ ■ ,Vj-i,Vi) - 9j+i(yo,yi, ■ ■ ■ ,Vj-i,Vj)) 

+ / K j (y j - 1 ,dy j )g j+1 (y ,y 1 ,...,y j _i,y j )- / 7r,(y.,_i, dy^+i^o, y x , . . . , y,,) 
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and consequently 

\9j{yo, yi,---, Vj-i) - 9j(yo, yi, ■ ■ ■ , < / ^(yj-i, d%) 

+ K-P fcj '«j+i, - ( pfej ^+i>^)l < L p{yj-i,yj-i) + Cj£j+i,jp(yj-i,yj-i), 

where 9 = 07 if j odd, 9 = ey 110 ' 1 if j even and g j+ i(-) = g j+1 (y ,yi, ■ ■ . ,%- 2 ,%_i, •)• 
Hence we have 

< L + Cj£j + ij for j = 1, . . . , 2(/c + n) — 1. 

Since £ 2 (fc+n),2(fe+n)-i < an easy computation shows that 

L(c 7 + 1) 

^1,0 S — : • 

1 - 7o 

This completes the proof. □ 

3. The law of the iterated logarithm. 

3.1. A martingale result. We start with recalling a classical result due to C.C. Heyde 
and D.J. Scott [9]. Let {S n , T n : n > 0} be a martingale on the probability space (Q, J 7 , P) 
where F Q = {fl, 0} and T n is the a-field generated by Si, S 2 , ■ ■ ■ , S n for n > 0. Let 
So = Z = P-a.s. and S n = J2k=i for n > 1. Further, let s 2 = ES^ < 00. 

We consider the metric space (C, p) of all real- valued continuous functions on [0, 1] with 

p(x,y) — sup \x(t) — y(t)\ forx,yGC7. 

0<t<l 

Let K be the set of absolutely continuous functions x G C such that x(0) = and 

r V(t)) 2 dt < 1. 

Define the real function g on [0, 00) by g(s) = sup{n : < s}. We define a sequence of 
real random functions r\ n on [0,1], for n > g{e), by 

Sk + (s 2 n t-sl)(sl +1 -sl)^Z k+ i 



Vn(t) 



^/2s 2 n log log s] 



if s 2 k < s 2 n t < s 2 k+1 , k = l,...,n-l and 



77„(£) = for n < #(e). 

Proposition 1. (Theorem 1 in [5]) If — > 00 and 

00 

(3.1) ^ s- 4 E[Z^l { | Zn | <7Sn} ] < 00 /or some 7 > 0, 

n=l 
oo 

(3.2) ]T s^EP^Il^i^j] < 00 for all e > 0, 

n=l 
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(3.3) s~ 2 ^Zl^l F-a.s.asn^oo 

k=l 

hold, then {r/ n } n >i is relatively compact in C and the set of its limit points coincides with 
K. 

3.2. Application to Markov chains. Let ip : E — > M be a Lipschitz function such 
that = 0, otherwise we could consider if) — -0 — Let L > denote its 

Lipschitz constant. Let (X n ) n >o be a Markov chain with the initial distribution /i satisfying 
conditions (H0)-(H3). 

We have £"ol P V(:r)| = £~ |<^<U>*> - < cd(5 x , ^) £~ T j < oc, by 

(H2). Thus we may define the function 



:= ' S ^ / P l il){x) for x E E. 



i=0 



We easily check that x is a Lipschitz function. 
It is well known that 



S n = x(X n ) - X(X ) + ^ W iorn>0 

i=0 

is a martingale on the space (E®, B(E)® N , P^) with respect to the natural filtration and its 
square integrable martingale differences are of the form 

Z n = x(X n )- X (X n . 1 ) + iP(X n ) forn>l. 

Observe that 7 V , . Z\ < oo. Indeed, we easily check that x — > E X (Z 2 A k) for any k > 1 is 
a bounded continuous function. Further, since E M pn(Z 2 A /c) = j E K x (Zf A k)fiP n (dx) — )■ 
E^ (Z 2 A fc) for any & > 1 as n — > oo and sup n>0 E M p?i (Z\ ) < oo, we obtain that E M>( (Z 2 ) < 
oo. 
Set 

a 2 :=E^Z 2 . 

We have 

(3.4) lim EuPnZl = lim E„Z 2 = a 2 . 

n—>oo n—^oo 

In fact, since x an d ip are Lipschitzean, we have sup n>1 E^\Z n \ 2+s < oo, by (H3). Further, 
observe that 

supE„(^l { | ZB | a > fc} ) < k~ s/2 su P E M |Z n | 2+<5 -> 

n>l n>l 

as fc —7- oo. Therefore, condition (13.41) follows from the fact that E M pn(Z 2 Ak) — >■ E M ,(Z 2 A£;) 
as n — ?• oo for any fc > 1. Finally, we obtain 

lim ^ = lim ^ = lim = a \ 

n— >oo 77, n— >oo 77, n— >oo 77, 
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Lemma 2. The square integrable martingale differences (Z n ) n >i satisfy the following con- 
dition: 

1 n 

(3.5) — > Z? — > a 2 Fn-a.s. as n — >■ oo 

n ' 

i=i 

and consequently if a 2 > condition A3. 3\) holds. 

Proof. First observe that to finish the proof it is enough to show that for any i 6 {1, . . . , no} 
we have 



1 

If we show that both the functions 



i +lno -> a 2 P M -a. 



x -> E,(| hminf (1/n ]T Z? +ino ) - a 2 | A 1] 

=1 



and 

n 

x -> Exd limsup(l/n^ Z? +Jno ) - a 2 | A 1) 
are continuous, we shall be done. Indeed, then we have 

n „ n 

E„(| limmf(l/nJ2Zi+in ) — o~ 2 \ A 1) = / E x (| liminf(l/n ]T Z 2 +l J - a 2 \ A l)u(dx 

1=1 J ^ 1=1 

„ n n 

= / E x .(|liminf(l/n]T^J-a 2 |A^ 

Jbj 1=1 1=1 



i+lno ) 17 



as m — > +oo, by the fact that jj l P l + mn o converges weakly to as m — > +oo. On the other 
hand, from the Birkhoff individual ergodic theorem we have 



it 



E M .(| hminf (1/n J2 Z lm ) ~ A A 1) = 

1=1 

and consequently 

n 

E„(| hminf (1/n J - a 2 | A 1) = 0, 

1=1 

which, in turn, gives 

n 

liminf (1/n V Z 2 +l ) = a 2 P M -a.s. 

n— >co ' * 

1=1 

Analogously we may show that 

n 

lim sup(l/ n ^ Z i+irJ = ^ P M -a.s. 



AN INVARIANCE PRINCIPLE... 7 

The remainder of the proof is devoted to showing the continuity of the relevant functions. 
Again, we restrict to the first function, since the proof for the second one goes in almost 
the same manner. 
Observe that 



1=1 



E x (|liminf(l/n^^J-a 2 |Al) 

1 n 



lim lim E 2 

n— >oo k—^oo 



mm 



i+lno ® i ■ ■ ■ i 



1 n+k 



n + k 



i+lng 



— a 



1=1 



A 1 



lim lim H„ u(x 



n— >oo k— >oo 



where 



H n>k {x) = E x (\ min{l/n(^ Z 2 +lno A n(l + a 2 )) -a 2 ,..., 

i=i 

n+k 

l/(n + ^n A (n + k)(l + a 2 ) - a 2 )}\ A 1) 

i=i 

n 

= E x (\ mm{l/n(J2(x(X i+ i m ) ~ XpQ-i-Mn ) + ^{X i+lno )f A n(l + a 2 )) -a 2 ,..., 
i=i 

n+k 

l/(n + k)(J2(x(Xi+in ) - x(Xi-i+ln ) + ^(*i+Jno)) 2 A (n + fc)(l + a 2 ) - <7 2 )}| A 1). 



2=1 



Set 



9n,k{yi,-- ■ ,2/2(n+fc)) = I min{l/n(J^(x(y 2 /) - xfe/ai-i) + "V>(2/2z)) 2 A n(l + a 2 )) - a 2 , . . . 

n+fc 

l/(n + fc)(X)(x(lto) - + <AM) 2 A (n + fc)(l + a 2 ) - a 2 )}\ A 1 



so that 

H nt k\X) = ¥, x [g n fi(Xi +no -i, Xi +nQ , Xi + 2n -l, Xi + 2n , ■ ■ ■ , X i+ 2(n+k)n -ly X i+ 2( n +k)n ) ) • 

Observe that H n ^ is given by formula (12.31) . If we show that there exists L such that 
g ns k is Lipschitz continuous in each variable with the Lipschitz constant L (independent 
of n, k), then all H n ^ are Lipschitzean with the same Lipschitz constant L, by Lemma 1. 
Consequently lim^oo lim^oo H n k is Lipschitzean and in particular continuous. Since min- 
imum of any finite family of functions which are Lipschitz continuous in each variable with 
the Lipschitz constant L is Lipschitz continuous in each variable with the same Lipschitz 
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constant L, to finish the proof it is enough to observe that the function 



{y u . . . , y 2p ) -)• l/p(%2{x{V2i) ~ x{V2i-i) + HV2i)) 2 A p(l + a 2 )) - a 1 



(Xmi) - Xmi-U WKm)) A jh± f a - r 2 
i=l 

is Lipschitz continuous in each variable with the Lipschitz constant L for fixed L > 0. On 
the other hand, each term in the above sum is Lipschitz continuous in each variable with 
the Lipschitz constant 

(l/p)(Lip x + Lip if))2p(l + a 2 ) = 2(Lip x + Lip ^)(1 + a 2 ). 

Observe that each variable appears in one term in the above sum. Hence L < 2(Lipx + 
Lip ■?/;)( 1 + <j 2 ), which finishes the proof. □ 

Note that following the proof of our previous lemma we are able to show that the 
considered Markov chain satisfies the strong law of large numbers (SLLN). This result 
however directly follows from Theorem 2.1 in [19]. 



Lemma 3. Let a 2 > 0. Under the assumptions (H0)-(H3) the square integrable martingale 
differences (Z n ) n >i satisfy conditions 113. A3.2\) . 

Proof. Since sup n>1 E M |Z n | 2+<5 < oo, 5 is the constant given in (H3), we have 

oo oo 

E s n%[^ 4 i { iz„i< 7 ,„ } ] < E s nV-^-%i^i 2+5 < i 2 - s su P E,\z n \ 2 + s j2 s 



oo 

2-5 

n 



1 i n>l , 

n=l n=l — n=l 

On the other hand, the condition s 2 Jn — > a 2 as n — >• oo gives Y^n=i s n. 2 ~ 5 < 00 > which 
completes the proof of condition (13. ip . 
To show condition (13.21) observe that 



Y,^[\Zn\l { \Z n \>,s n} ] <J2 S n 1 ^[\Zn\ 2+5 /(eS n ) 1+5 ] < 6^ SU P E„ | Z n | 2+5 ]T S~ 2 ~ S < OO. 
n=l n=l — n=l 

The proof is complete. □ 

3.3. The Law of Iterated Logarithm for Markov chains. 

Theorem 1. Let (X n ) n >o be a Markov chain with an initial distribution \i satisfying con- 
ditions (H0)-(H3). If if) is a Lipschitz function with (ip,fi*) = and a 2 > 0, then F^-a.s. 
the sequence 

e , t) = ElM^) + (nt - k)^(X k+1 ) 
a^2n log log ?2 

if k < nt < k + 1, k — 1, . . . , n — 1 for t > 0, n > e and n (t) = otherwise is relatively 
compact in C and the set of its limit points coincides with K . 

Proof. First observe that since s 2 n jn — > a 2 > as n — > oo we have 



— >• 1 as n — > oo. 

ay2n log log n 
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Consequently, from Lemmas 2 and 3 it follows that the sequence 

Vn{t) = 

ay/ In log logn 

if s 2 < s\t < Sfc +1 , k = 1, . . . , n — 1 for t > 0, n > e and ?7n(t) = otherwise is relatively 
compact in C and the set of its limit points coincides with K, due to Heyde and Scott [9]. 
Let t G (0, 1] and n > 1. Observe that if k < nt < k + 1, then 



2 



fca 2 2 nff 2 2 (fc + ]> 

e 2 b k ^ 2 6 ™ — e 2 b k+l- 



Set 

Wj = /oil 

<j\J In log log n 

where fc > 1 such that fc < nt < A; + 1. Since (ncr 2 ) /s 2 — >• 1 as n — > oo for any e > holds 

(1 - e)sl < (1 + e)s 2 n t < (1 + e) 2 (l - e)" 1 ^ 

for all n large enough. Hence there is t* G [t(l — e)(l + e)~ l ,t{l + e)(l — e)" 1 } such that 
s 2 < s 2 t* < On the other hand, the diameter of the interval [sf/s 2 , s^ +1 /s 2 ] for a 

fixed = 1, . . . , n — 1 converges to as n — > oo. Consequently, for any t > and n > e 
there exists t n > such that fj n (t) = r) n (t n ) and t„ — > t as n — > oo. Since the sequence 
{Vn{t))n>e is relatively compact in C and the set of its limit points coincides with K, the 
sequence (fj n (t)) n>e is also relatively compact and has the same set of limits points. 
Fix e > 0. Define the sets 

A n = (u,en: > e /2l uLn:' 2 '- ^ > »M &*»>!. 



n \/n 



Now we are going to show that X/nLi EV(Ai) < oo. Indeed, keeping in mind that x is 
Lipschitzean, by the Chebyshev inequality we obtain 



n I \ \/n 



E(p X0 (X n )y+ s + E(p Xi) (X ))^ s 

^ C . I-l.A/9 - 



n l+5/2 — n l+5/2' 

by (H3) for some constant c > independent of n. 

Analogously, we may check that there exists a positive constant C (independent of n) 
such that 

P, ({a, 6 fl = |Z " > «/2}) = P, ({„ 6 n : '*<*•> ^f^ 1 > «/2 

C 

< 



n l+5/2 ' 

by (H3) and the Lipschitz property of the function Thus the series P M (A n ) is 

convergent. 
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Finally, from the Borel-Cantelli lemma it follows that P„-a.s. 



limsup sup 

n— >oo 0<t<l 



S k + (nt - k)Z k+1 YLi M x i) + ( nt ~ k )^( X kA 



<jy/2n\og logn <jy/2n log log n 

where k < nt < k + 1. Since e > was arbitrary, the proof is complete. □ 



< e, 
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